1. Field of the Invention
The present invention relates to a picture processing apparatus for detecting a moving vector of a moving picture.
2. Description of the Related Art
To eliminate noise from a picture, a motion adaptive type recursive filter has been used. An example of the structure of the motion adaptor recursive filter is shown in FIG. 1. Picture data of a considered frame (namely, an input frame at each time point) is supplied to a subtracting circuit 1100. Picture data of the preceding frame (one frame earlier than the considered frame) is supplied from a frame memory 1099 to the subtracting circuit 1100. An output of the subtracting circuit 1100 is supplied to an absolute value calculating circuit 1101. The absolute value calculating circuit 1101 converts the output data of the subtracting circuit 1100 into an absolute value. The absolute value is supplied to a threshold value processing circuit 1102. The threshold value processing circuit 1102 compares the supplied absolute value with a predetermined threshold value so as to determine whether the picture moves or stops for each pixel.
The determined result of the threshold value processing circuit 1102 is supplied to a weight designated value generating circuit 1103. The weight designated value generating circuit 1103 designates a weight designated value k corresponding to the supplied determined result. The weight designated value k is supplied to an amplifier 1105. The amplifier 1105 adjusts the amplitude of the input frame. An amplifier 1104 amplifies the input signal by k times. On the other hand, the amplifier 1105 amplifies pixel date stored in the frame memory by (1xe2x88x92k) times.
When the threshold value processing circuit 1102 has determined that the picture of the current frame stops, a fixed value ranging from 0 to 0.5 as the value k is designated. Thus, an output of an adder 1106 disposed downstream of the amplifiers 1104 and 1105 becomes a weighted value of a pixel value of the considered frame and a pixel value of the preceding frame. On the other hand, when the threshold value processing circuit 1102 has determined that the picture of the current frame moves, xe2x80x9c1xe2x80x9d is designated to the value k. As a result, the adder 1106 directly outputs a pixel value of the considered frame.
The above-described motion adaptive type recursive filter has problems (a) to (d) that follow. (a) Since a still portion is weighted with the fixed value k, if the intensity and/or direction of noise varies, the noise cannot be adequately eliminated. (b) When the noise level is large, a still portion is mistakenly determined as a moving picture. Thus, the noise eliminating effect deteriorates. (c) A moving portion may be mistakenly determined as a still picture. As a result, deterioration such as an occurrence of a trailing shape portion may take place. (d) Noise cannot be eliminated from a moving portion.
In addition, a noise eliminating method using a class category adaptive process has been proposed as Japanese Patent Application No. 10-304058. In the method, a block matching operation is performed so as to detect a moving vector. Corresponding to the detected moving vector, a class category adaptive process using pixels at the same position on frames is performed. Although the problems of the motion adaptive type recursive filter can be solved or alleviated using the method, it has the following problems.
(e) Since the block matching operation is performed, a heavy load is imposed to the apparatus. (f) The block matching operation should be performed between two adjacent frames (for example, between the frames t and (t+1), between the frames (t+1) and (t+2), and so forth). If the block matching operation fails for at least two adjacent frames, the noise elimination effect deteriorates. (g) In the block matching operation, a motion of a small object in a block cannot be detected. Thus, such a motion may result in an inadequate process.
In addition, as other related art references rather than the motion adaptive type recursive filter for detecting a moving vector, block matching method, slope method, and so forth are known. In the block matching method, the sum of differences of pixels of each block in each searching range should be calculated and compared. Thus, the amount of calculation becomes large. Consequently, there are problems from view points of the circuit scale, the process time, and so forth. In addition, a small motion in a block cannot be accurately detected. In the slope mode, if a picture contains noise, the detection accuracy of the moving amount remarkably deteriorates.
Next, the above-described related art reference (Japanese Patent Application No. 10-304058 and corresponding U.S. patent application Ser. No. 09/602,934 filed on Oct. 25, 1999) that had not been disclosed on the filing date of the prior Japanese application of the present patent application will be described in detail. FIG. 2 shows an example of the structure of the related art reference. As was described above, in the related art reference, in a combination of the block matching operation and the class category adaptive process, noise is eliminated from a moving picture. An input picture signal from which noise should be removed is supplied to a frame memory 200. Frame memory 201 and 202 are connected in a shift register shape to the frame memory 200. Thus, picture data of three chronologically successive frames are stored to the frame memories 200, 201, and 202.
The frame memories 200 and 201 supply stored frames to a moving vector detecting portion 203. The moving vector detecting portion 203 performs a block matching operation for the two supplied frames and detects a moving vector. The moving vector detecting portion 203 supplies the detected moving vector to area extracting portions 205 and 206. In addition, the frame memories 200, 201, and 202 supply their stored frames to the area extracting portion 205 and 206. A moving vector detecting portion 204 performs the block matching operation corresponding to the supplied two frames and detects a moving vector. The moving vector detecting portion 204 supplies the detected moving vector to the area extracting portions 205 and 206.
The frames stored in the frame memories 200, 201, 202 are supplied to the area extracting portion 206. The area extracting portion 206 references the moving vector supplied from the moving vector detecting portion 203 and the moving vector supplied from the moving vector detecting portion 204 and extracts a picture area at a predetermined position from the supplied frame and supplies data of the extracted picture area to a class code generating portion 207. The class code generating portion 207 extracts a space-time pattern of pixel values corresponding to for example ADRC (Adaptive Dynamic Range Coding) method and generates a class code that represents a class categorized corresponding to the extracted pattern.
Since the picture area that is extracted by the area extracting portion 206 is used for a class category process, such a picture area is referred to as class tap. The class code is supplied to a coefficient memory 208. The coefficient memory 208 stores a predictive coefficient set for individual classes. The coefficient memory 208 outputs a predictive coefficient corresponding to a class code. An output of the coefficient memory 208 is supplied to an estimation calculating portion 209.
On the other hand, picture data of frames stored in the frame memories 200, 201, and 202 are supplied to the area extracting portion 205. The area extracting portion 205 references moving vectors supplied from the moving vector detecting portions 204 and 203, extracts picture areas from the picture data of the supplied frames, and supplies data of the extracted picture areas to the estimation calculating portion 209. The estimation calculating portion 209 performs a predetermined calculation corresponding to the data supplied from the area extracting portion 205 and predictive coefficient set supplied from the coefficient memory 208 and generates a resultant output picture. Noise has been eliminated or alleviated in the output picture. The picture area extracted from the area extracting portion 205 is used for a calculation for predictively generating an output picture. The picture area is referred to as class tap.
Next, with reference to FIGS. 3A to 3E, the tap structures of a class tap and a predictive tap that are extracted by the area extracting portions 206 and 207 will be described. In FIGS. 3A to 3D, a considered pixel that is predicted is denoted by a black circle. In addition, pixels extracted as class taps or predictive taps are denoted by shaded circles. FIG. 3A shows an example of a basic class tap structure. Pixels at the same spatial position of the considered frame f [0] containing the considered pixel, the immediately earlier frame f [xe2x88x921] and the immediately later frame f [+1] are extracted as class taps. FIG. 3B shows an example of a basic predictive class tap structure. A total of 13 pixels of a considered pixel and 12 pixels surrounding the considered pixel are extracted as predictive taps from picture data of a considered frame and picture data of frames at the immediately earlier and later frames of the considered frame.
FIGS. 3C and 3D show that pixels that are extracted are chronologically moved corresponding to moving vectors that are output from the moving vector detecting portions 203 and 204. As shown in FIG. 3E, when the moving vector of a considered frame is (0, 0), the moving vector of the immediately earlier frame is (xe2x88x921, xe2x88x921), and the moving vector of the immediately later frame is (1, 1), the extracting positions of the class taps and the predictive taps of all those frames are parallel-translated corresponding to the moving vectors. Thus, according to the structure of the related patent application, before extracting class taps and predictive taps, it is necessary to detect moving vectors. In that case, the block matching operation for detecting moving vectors requires a large amount of calculation. Thus, a large load is applied to the apparatus.
Therefore, an object of the present invention is to provide a processing apparatus and a processing method for accurately calculating a moving vector of a moving picture.
A first aspect of the present invention is a picture processing apparatus, comprising an extracting means for extracting a plurality of pixels on each straight line that extends in the chronological direction and that passes through each of objective pixels including a considered pixel of a picture signal and a plurality of pixels in the vicinity of the considered pixel, a similarity detecting means for detecting the similarity in each of the straight lines according to the pixel values of a plurality of pixels extracted by the extracting means in each of the straight lines for each of the objective pixels against the considered pixel, and a moving vector calculating means for calculating a moving vector of the considered pixel according to the similarity detected in each of the straight lines for each of the objected pixels.
A second aspect of the present invention is a picture processing apparatus, comprising an extracting means for extracting a plurality of pixels on a particular straight line that passes through a considered pixel of a picture signal and that extends in the chronological direction and on each of straight lines parallel to the particular straight line that pass through pixels in the vicinity of the considered pixel, a similarity detecting means for detecting similarity in the direction of each of the straight lines for the plurality of pixels extracted by the extracting means in the direction of each of the straight lines, and a moving vector calculating means for calculating a moving vector of the considered pixel according to the similarity detected in the direction of each of the straight lines.
A third aspect of the present invention is a picture processing method, comprising the steps of (a) extracting a plurality of pixels on each straight line that extends in the chronological direction and that passes through each of objective pixels including a considered pixel of a picture signal and a plurality of pixels in the vicinity of the considered pixel, (b) detecting the similarity in each of the straight lines according to the pixel values of a plurality of pixels extracted at the step (a) in each of the straight lines for each of the objective pixels against the considered pixel, and (c) calculating a moving vector of the considered pixel according to the similarity detected in each of the straight lines for each of the objected pixels.
A fourth aspect of the present invention is a picture processing method, comprising the steps of (a) extracting a plurality of pixels on a particular straight line that passes through a considered pixel of a picture signal and that extends in the chronological direction and on each of straight lines parallel to the particular straight line that pass through pixels in the vicinity of the considered pixel, (b) detecting the similarity in the direction of each of the straight lines for the plurality of pixels extracted at the step (a) in the direction of each of the straight lines, and (c) calculating a moving vector of the considered pixel according to the similarity detected in the direction of each of the straight lines.